Skip to content

feat: Add consolidated snippets.#396

Draft
kinyoklion wants to merge 5 commits intomainfrom
rlamb/sdk-snippets
Draft

feat: Add consolidated snippets.#396
kinyoklion wants to merge 5 commits intomainfrom
rlamb/sdk-snippets

Conversation

@kinyoklion
Copy link
Copy Markdown
Member

@kinyoklion kinyoklion commented Apr 24, 2026

First slice of snippets/: a self-contained Go module that owns the canonical source for LaunchDarkly SDK code samples and renders them into downstream consumers. Scope: python-server-sdk "Getting Started" flow, end-to-end.

This is an MVP just for the getting started snippets. Later PRs will cover other documentation situations.

In scope

  • Snippet file format (YAML frontmatter + one fenced code block).
  • Generator CLI (render, verify, validate).
  • ld-application adapter — rewrites the body of marked <Snippet> elements in TSX.
  • Docker validator that runs the snippet verbatim against a real LD environment.

Out of scope (later slices)

  • All SDKs other than python-server-sdk.
  • ld-docs adapter, GitHub Actions, signed-binary release, sdk-meta capability integration, region/version conditionals.

Snippet file

---
id: python-server-sdk/getting-started/install
sdk: python-server-sdk
kind: install
lang: shell
inputs:
  version: { type: string, runtime-default: "" }
---

```shell
echo "launchdarkly-server-sdk{{ if version }}=={{ version }}{{ end }}" >> requirements.txt && pip3 install -r requirements.txt

Templating: `{{ name }}` substitutes; `{{ if name }}…{{ end }}` is a non-empty conditional. The `ld-application` adapter renders these as JS template-literal expressions: `${name}` and `${name ? \`…\` : ''}`.

## Render markers

Markers sit in the host file's own comment syntax. The generator replaces only the children of the JSX element that follows. Marker hash + version are filled in by `snippets render`.

```tsx
{/* SDK_SNIPPET:RENDER:python-server-sdk/getting-started/install hash=8e6373549d36 version=0.1.0 */}
<Snippet lang="shell" withCopyButton>{`echo "launchdarkly-server-sdk${version ? `==${version}` : ''}" >> requirements.txt && pip3 install -r requirements.txt`}</Snippet>

verify recomputes the hash and fails on hand-edits inside marked regions.

CLI

# Rewrite all marked regions in a consumer checkout
snippets render --target=ld-application --out=/path/to/app

# Fail if anything drifted (intended for consumer CI)
snippets verify --target=ld-application --out=/path/to/app

# Run the snippet in Docker against a real LD environment
export LAUNCHDARKLY_SDK_KEY=…  
export LAUNCHDARKLY_FLAG_KEY=…
snippets validate --sdk=python-server-sdk

Test plan

  • cd snippets && go build ./... && go test ./... clean.
  • snippets render --target=ld-application --out=<app> is idempotent on a checkout that already contains the markers.
  • snippets verify rejects a hand-edit inside a marked region.
  • snippets validate --sdk=python-server-sdk against a real LD test env prints *** The <flag> feature flag evaluates to <value> and exits 0.

@kinyoklion kinyoklion changed the title Rlamb/sdk snippets feat: Add consolidated snippets. Apr 24, 2026
@kinyoklion kinyoklion closed this Apr 25, 2026
…ing-started

Introduces sdk-meta/snippets/, a self-contained Go module that owns the
canonical source for LaunchDarkly SDK code samples and renders them
into downstream consumers. Slice scope: python-server-sdk "Getting
Started" flow, end-to-end.

What's here
-----------
- cmd/snippets — CLI with three subcommands:
    render   — rewrite the body of every JSX element marked with an
               SDK_SNIPPET:RENDER comment in a consumer checkout
    verify   — recompute hashes and fail if any managed region drifted
    validate — run each snippet inside a per-language Docker validator
- internal/model — snippet file format (YAML frontmatter + one fenced
  code block) loaded from sdks/<id>/snippets/.
- internal/render — minimal {{ var }} / {{ if var }}...{{ end }}
  templating engine with a runtime mode for validation and a JS
  template-literal mode for the ld-application adapter.
- internal/markers — host-syntax-aware scanner for SDK_SNIPPET:RENDER
  comments (`// …`, `{/* … */}`, `/* … */`); hashes the rendered region
  to catch hand-edits.
- internal/adapters/ldapplication — first adapter target. Discovers
  consumer files via each sdk.yaml's ld-application.get-started-file
  field and rewrites only the JSX children of marked <Snippet>
  elements, preserving surrounding whitespace.
- internal/validate — orchestrator that builds a Docker image per
  language and runs the snippet verbatim against a real LaunchDarkly
  environment. The SDK key and flag key come from the caller's
  LAUNCHDARKLY_SDK_KEY / LAUNCHDARKLY_FLAG_KEY env vars (the same
  convention the hello-* sample apps use), are forwarded into the
  container, and never end up in committed files.
- sdks/python-server-sdk — sdk descriptor plus four snippets sourced
  verbatim from the existing get-started flow:
    getting-started/mkdir
    getting-started/install
    getting-started/main-py
    getting-started/run
- validators/languages/python — Docker image plus run.sh that pip-
  installs the snippet's own requirements, runs the entrypoint, and
  matches the expected flag-evaluation line within a timeout.
- docs/AUTHORING.md — snippet authoring guide.

Naming and module path
----------------------
- Go module: github.com/launchdarkly/sdk-meta/snippets
- Adapter target: --target=ld-application (renders into the LD
  application UI). A future --target=ld-docs adapter is planned for
  the docs site; not implemented yet.

Out of scope (kept in design doc, not in this commit): all SDKs other
than python-server-sdk, the ld-docs MDX adapter, GitHub Actions,
signed-binary release pipeline, sdk-meta capability integration,
region/version conditional rendering.

Verified locally
----------------
- go build ./...           : clean
- go test ./...            : render + markers tests pass
- snippets render --target=ld-application --out=<app-checkout>: idempotent
- snippets verify --target=ld-application --out=<app-checkout>: ok
- snippets validate (with LAUNCHDARKLY_SDK_KEY + LAUNCHDARKLY_FLAG_KEY):
  matches "*** The <flag> feature flag evaluates to ..." against a
  real LD environment.
@kinyoklion kinyoklion reopened this Apr 25, 2026
Findings 1-19 from the multi-agent review (artifacts/multi-review-sdk-meta-396.md).
All worth-fixing items are addressed in this commit; the four "could not
prove" hardening items are deferred (they remain open documentation in
the review file).

Security and correctness
------------------------
- #1, #5: Marker hash now covers the full <Tag …>…</Tag> element, not just
  the children, so attribute-only edits (e.g. lang="python" → lang="go")
  are detected by `verify`. The `hash=` field is required at verify time;
  a missing hash is an error rather than a skip.
- #2: validation.entrypoint must be a plain filename (filepath.Base equal,
  no path separators or ..). Blocks the "snippet writes to ~/.ssh" class
  of attack via author-controlled YAML.
- #3: ld-application.get-started-file is rejected if it's absolute or if
  filepath.Rel(appDir, full) starts with "..". Blocks the same class via
  the consumer-side path.
- #4: Template tokenizer was treating any token starting with "end"
  (`endTime`, `endIndex`, …) as `{{ end }}`. Switched HasPrefix → equality.
- #6: Marker scanner now tracks <Tag depth so a nested same-tag pair
  (`<Snippet><Snippet>…</Snippet></Snippet>`) doesn't silently truncate
  to the inner close. Same-prefix tags (`<SnippetGroup>`) don't count
  as opens of `<Snippet>`.
- #7: runtimeInputs has a `sdk-key` arm wired to LAUNCHDARKLY_SDK_KEY,
  with a defensive check that flag-key/sdk-key inputs cannot declare a
  runtime-default (those values must always come from the environment).
- #8: validation.requirements rejects newlines and lines starting with
  `-` so a snippet can't smuggle `--extra-index-url` through to pip.
- #9: Backtick-string scanner now tracks `${ … }` expression depth, so a
  nested template literal inside an interpolation expression doesn't
  prematurely end the outer string.
- #10: Render path split into RenderForLDApplicationTemplate (escaping
  for backtick literals) and RenderForJSXText (no escaping; for bare
  text). The bare-text path no longer corrupts backslashes / backticks
  in user-visible output.
- #11: Atomic write — temp file in the same directory, fsync, rename;
  source file mode is preserved.
- #12: Validator Docker tag is a content hash of the validator dir, so
  concurrent runs against the same validator share the cached image and
  runs against different validators cannot interleave.
- #13: run.sh redacts LAUNCHDARKLY_SDK_KEY from the log dump on failure.
- #14: Bare-vs-backticks decision is driven by the snippet's intent
  (interpolation / multiline / JSX-special chars), not by what's already
  in the file. Re-renders no longer stay sticky on the wrapped form.
- #17: snippet frontmatter and sdk.yaml are decoded with KnownFields(true)
  so a typo like `Entrpoint:` is a hard error.

Cosmetic / docs
---------------
- #15: AUTHORING.md notes the uppercase-first JSX-component-tag constraint.
- #16: hashLen const + comment documenting the 12-hex-char (~48 bit)
  budget is for accidental-drift detection only, not integrity.
- #19: go.mod uses `go 1.24` (drops the patch version).

Tests
-----
- New tests for: endFoo regression, unmatched/unclosed `{{ if/end }}`,
  unknown variable, empty template, RenderForJSXText backslash round-trip,
  HasInterpolation, ContainsJSXSpecial, scanner edge cases (no markers,
  block-style, unterminated comment, gap, self-closing, missing close,
  nested same-tag, similar-prefix tag, nested-backtick), full-element
  hash detects attribute edits, descriptor-traversal rejects, runtimeInputs
  rejects runtime-default on key types, requirements rejects pip flags,
  entrypoint rejects path components.

Verified end-to-end
-------------------
- go build ./... / go vet ./... / go test ./...     all clean
- snippets render --target=ld-application --out=<app>: rewrites the file
  with full-element hashes; second run reports "no changes".
- snippets verify ok on the freshly rendered file.
- snippets verify rejects: a one-byte attribute edit (lang="python" →
  lang="go") AND a marker with the hash= field stripped.
- snippets validate (with real LAUNCHDARKLY_SDK_KEY / LAUNCHDARKLY_FLAG_KEY)
  matches the expected flag-evaluation line.
Walk back the part of the prior review-feedback commit that hashed the
full <Tag …>…</Tag> region. The scope=content contract says the
consumer owns the element's attributes; locking them down forces a
re-render every time someone tweaks `withCopyButton` / `label="…"` /
`className` and offers nothing the design promised.

`verify` now:
- requires a hash= field on every marker (#5, unchanged)
- compares it against the SHA-256 of src[RegionStart:RegionEnd] (just
  the children, as originally documented)
- accepts attribute-only edits, rejects child edits

Tests updated: TestVerify_AcceptsAttributeEdit and
TestVerify_RejectsChildEdit pin the new contract; the dead
TestFullElementHash_DetectsAttributeEdit is removed; the now-unused
Match.FullElementHash method is dropped.

End-to-end re-checked: render is idempotent, verify ok, an attribute
edit passes verify, a child edit fails verify with a children-hash
error, and `snippets validate` against a real LD environment still
prints `*** The sample-feature feature flag evaluates to True`.
Brings the generator-side improvements developed on the
port-remaining-getting-started branch back to this branch so the
tooling lives with the original framework slice. The port branch will
rebase on top of this and carry only snippet content + validators +
CI.

What lands here (no new SDKs or validators outside python):

- internal/validate/runner.go (new) + restructured validate.go.
  Per-runtime dispatcher driven by validators/languages/<runtime>/runner.yaml
  (mode: docker | native, runs-on hint, image-prefix). Snippets pick a
  validator via `validation.runtime`; the dispatcher stages the snippet
  body plus any `validation.companions:` files at their `file:` paths,
  then either docker-builds + runs the language image or execs the
  harness directly. Build context is now the entire `validators/` dir
  so each Dockerfile can pull from `shared/` alongside its own
  `languages/<runtime>/`.

- envInputs covers the full EXAM-HELLO env-var set
  (LAUNCHDARKLY_SDK_KEY / FLAG_KEY / MOBILE_KEY / CLIENT_SIDE_ID).
  runtimeInputs has arms for sdk-key, flag-key, mobile-key, and
  client-side-id; declaring runtime-default for any of those is an
  error. requireEnvForInputs fails fast with a clear message before a
  pip install or docker build burns time.

- internal/model/model.go: Validation gains Runtime + Companions
  fields. Frontmatter still loaded with KnownFields(true).

- internal/render/render.go: render functions now take a declared-
  inputs set. `{{ name }}` for a name not in the set passes through
  as literal `{{ name }}` so foreign template syntax (e.g. Vue's
  `{{ flagValue }}` mustaches in a Vue snippet body) survives both
  runtime substitution and ld-application rendering. Conditionals
  still require declared inputs (Vue uses `v-if`, not `{{ if … }}`).

- internal/markers/markers.go: skipPlainString returns i+1 when no
  closing quote precedes the next newline — fixes the case where an
  apostrophe in JSX text (`SDK's shared libraries`) was eaten as the
  start of a string literal and the scanner skipped past following
  render markers. Multi-line strings use backticks (handled by
  skipBacktick), so this is safe.

- internal/adapters/ldapplication/ldapplication.go: threads the
  declared-input set through renderForJSXChild so the bare-vs-template
  decision and the foreign-template pass-through both work.

- validators/shared/lib.sh (new): shared harness helpers
  (require_env, await_success_line, dump_redacted, fail_with_log).
  Each per-language harness will source it for the polling/timeout
  loop and key-redacting log dump.

- validators/languages/python/: Dockerfile rebased on the new build
  context (COPY shared /harness-shared, COPY languages/python/harness
  /harness). harness/run.sh sources the shared lib. New runner.yaml
  declaring mode: docker.

Verified: go build / go test clean; snippets render --target=ld-
application --out=<gonfalon> against the python-only state on this
branch is idempotent and verify passes.
Comment on lines +17 to +18
snippets verify --target=ld-application --out=<app-checkout> [--sdks=./sdks]
snippets validate --sdk=<sdk-id> [--sdks=./sdks] [--validators=./validators]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know the difference between these two just looking at it. Maybe some description or we can land on different language once I've actually looked at the PR.

Comment thread snippets/docs/AUTHORING.md Outdated
|---|---|---|
| `id` | yes | globally unique; convention `<sdk>/<group>/<name>` |
| `sdk` | yes | matches a directory under `sdks/` |
| `kind` | yes | `bootstrap`, `install`, `hello-world`, `run` |
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that list exclusive and what's the distinction?

Comment thread snippets/docs/AUTHORING.md Outdated
- Block comment anywhere: `/* SDK_SNIPPET:RENDER:<id> hash=<h> version=<v> */`

`hash` and `version` are filled in by `snippets render`. On first wiring, use
`hash=000000000000` as a placeholder — the next render rewrites it.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not a terribly convenient number to remember to use. Possible to make it hash=0? I realize the bots will do fine with it, but I'm not totally ready to surrender a touch of humanity.

- main.go usage: spell out what each subcommand does inline so the
  difference between `verify` (read-only CI check) and `validate` (real
  LaunchDarkly e2e exercise) is clear without having to read the
  source. Also drop the stale "first-pass support: python-server-sdk"
  line; the matrix has expanded since.

- AUTHORING.md `kind`: list the four values out with examples and call
  out that the set is closed for now (one snippet has exactly one
  kind).

- AUTHORING.md hash placeholder: recommend `hash=0` instead of
  `hash=000000000000` — both work (the regex captures any hex string;
  the placeholder is just there to make the marker syntactically
  valid until the first render rewrites it) and `0` is friendlier to
  type by hand.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants